Facial Emotion Detection Using Convolutional Neural Networks and Representational Autoencoder Units

Emotion being a subjective thing, leveraging knowledge and science behind labeled data and extracting the components that constitute it, has been a challenging problem in the industry for many years. With the evolution of deep learning in computer vision, emotion recognition has become a widely-tackled research problem. In this work, we propose two independent methods for this very task. The first method uses autoencoders to construct a unique representation of each emotion, while the second method is an 8-layer convolutional neural network (CNN). These methods were trained on the posed-emotion dataset (JAFFE), and to test their robustness, both the models were also tested on 100 random images from the Labeled Faces in the Wild (LFW) dataset, which consists of images that are candid than posed. The results show that with more fine-tuning and depth, our CNN model can outperform the state-of-the-art methods for emotion recognition. We also propose some exciting ideas for expanding the concept of representational autoencoders to improve their performance. 1. Background and Related Works The basic idea of using representational autoencoders came from a paper by Hadi Amiri et al. (2016) and they used context-sensitive autoencoders to find similarities between two sentences. Loosely based on that, we expand that idea to the field of vision which will be discussed in the upcoming sections. There are works that used convolutional neural networks for emotion recognition. Lopes et al. (2015) created a 5-layer CNN which was trained on Cohn – Kanade (CK+) database for classifying six different classes of emotions. A lot of preprocessing steps such spatial and intensity normalization were done before inputting the image to the network for training in this method. Arushi and Vivek (2016) used a VGG16 pretrained network for this task. Hamester et al. (2015) proposed a 2-channel CNN where the upper channel used convolutional filters, while the lower used Gabor-like filters in the first layer. Xie and Hu (2017) proposed a different type of CNN structure that used convolutional modules. This module, to reduce redundancy of same features learned, considers mutual information between filters of the same layer, and processes the best set of features for the next layer.